Bayes Optimal Instance-Based Learning
نویسندگان
چکیده
In this paper we present a probabilistic formalization of the instance-based learning approach. In our Bayesian framework, moving from the construction of an explicit hypothesis to a data-driven instance-based learning approach, is equivalent to averaging over all the (possibly innnitely many) individual models. The general Bayesian instance-based learning framework described in this paper can be applied with any set of assumptions deening a parametric model family, and to any discrete prediction task where the number of simultaneously predicted attributes is small, which includes for example all classiication tasks prevalent in the machine learning literature. To illustrate the use of the suggested general framework in practice, we show how the approach can be implemented in the special case with the strong independence assumptions underlying the so called Naive Bayes classiier. The resulting Bayesian instance-based classiier is validated empirically with public domain data sets and the results are compared to the performance of the traditional Naive Bayes classiier. The results suggest that the Bayesian instance-based learning approach yields better results than the traditional Naive Bayes classiier, especially in cases where the amount of the training data is small.
منابع مشابه
Active Learning with Partially Labeled Data via Bias Reduction
With active learning the learner participates in the process of selecting instances so as to speed-up convergence to the “best” model. This paper presents a principled method of instance selection based on the recent bias variance decomposition work for a 0-1 loss function. We focus on bias reduction to reduce 0-1 loss by using an approximation to the optimal Bayes classifier to calculate the b...
متن کاملCalculation of the Learning Curve
The learning curve of Bayes optimal classii-cation algorithm when learning a perceptron from noisy random training examples is calculated exactly in the limit of large training sample size and large instance space dimension using methods of statistical mechanics. It is shown that under certain assumptions, in this \thermodynamic" limit, the probability of misclassiication of Bayes optimal algor...
متن کاملNaive Bayes for Regression
Despite its simplicity, the naive Bayes learning scheme performs well on most classiication tasks, and is often signiicantly more accurate than more sophisticated methods. Although the probability estimates that it produces can be inaccurate, it often assigns maximum probability to the correct class. This suggests that its good performance might be restricted to situations where the output is c...
متن کاملAttribute-Based Learning
1 Main approaches The following main approaches to attribute-based learning are considered in this paper: • Rule induction • Decision-tree induction • Regression-tree induction • Naive Bayes classifiers We have included Naive Bayes Classifiers, that is Bayes classifiers which assume that the attributes are independent, although they are not typically considered as an AI approach to Machine Lear...
متن کاملExploring a Framework for Instance Basedlearning and Naive Bayesian
The relative performance of diierent methods for classiier learning varies across domains. Some recent Instance Based Learning (IBL) methods, such as IB1-MVDM* 10 , use similarity measures based on conditional class probabilities. These probabilities are a key component of Naive Bayes methods. Given this commonality of approach, it is of interest to consider how the diierences between the two m...
متن کامل